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Abstract 

We present an integrated architecture for 
word-level and sentence-level processing in a 
unification-based paradigm. The core of the sys- 
tem is a CLP implementation of a unification en- 
gine for feature structures supporting relational 
values. In this framework an HPSG-style gram- 
mar is implemented. Word-level processing uses 
X2MorF, a morphological component based on 
an extended version of two-level morphology. This 
component is tightly integrated with the grammar 
as a relation. The advantage of this approach is 
that morphology and syntax are kept logically au- 
tonomous while at the same time minimizing in- 
terface problems. 

1 Introduction 

Over the last few years there has been a growing 
interest in computational morphology and phonol- 
ogy. A number of systems have been developed 
that deal with word-level processing. A widely 
used approach is finite-state morphology, most no- 
tably two-level morphology (for an introduction, 
see pproat ("92| )). Morphological components are 
successfully used for a wide range of stand-alone 
applications like spelling correction and hyphen- 
ation. One obvious application is the use in NLP 
systems geared to the analysis/generation of text. 
Surprisingly, they have not been widely applied in 
this domain up to now. 

A major reason for this is the problem of 
interfacing morphology with syntax. Reflecting 
the current trend in syntax towards lexicalism, 
unification-based systems use highly structured 
feature structures as input. Translating the out- 
put of morphological components into such a rep- 
resentation has proved to be difficult. Reducing 
interface problems is therefore crucial to success. 
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A tight integration between word and sen- 
tence level processing also has linguistic advan- 
tages. The boundary between morphology and 
syntax is fuzzy. When processing written text the 
units morphology has to deal with are, in a tech- 
nical sense, not words but character strings sepa- 
rated by delimiters. While these strings roughly 
correspond to the words of a sentence there are 
problematic cases. In German, e.g., zw-infinitive 
or verbs with separable prefixes are written as a 
single unit in some instances and separately in 
others. 

The problem has been recognized and some 
possible remedies have been proposed. They all 
try to minimize or to eliminate the interface be- 
tween word and sentence level processing. One 
step is the description of word formation in terms 
of a unification-based grammar to make the result 
of morphological processing directly available to 
syntax and vice versa, an approach already taken 
in X2MorF (p:rost, 90| ; p?rost, 9lD , an extension 
of two-level morphology. 

The harder problem is the integration of mor- 
phophonology which is traditionally formalized in 
a way not easily translatable into the feature for- 
malism. We will show how this can be achieved 
by merging the word-level grammar of X2MorF 
into an HPSG-style grammar, and by adopting a 
relational view of its two- level rules. 

In this paper we assume basic familiarity with 
unification-based NLP techniques and two-level 
morphology. 

2 Integrating Morphology into 

HPSG 

Head-driven Phrase Structure Grammar (HPSG, 
Pollard k Sag (871 ), jPollard fc Sag (94| )) can be 
viewed as a mono-level but multi-stratal theory of 
grammar, where different strata relate to differ- 
ent aspects of linguistic information, but are rep- 
resented uniformly in feature logics. As such it is 
well suited as a linguistic theory for our enterprise. 



HPSG differentiates between three strata — 
PHON, SYNSEM and DTRS. Though morphology is 
not considered in the standard approach, it sug- 
gests itself to be included as a fourth stratum by 
introducing a feature MORPH into the type sign. 
Morphotactics are easily described in terms of a 
feature based grammar. The problem is how to 
deal with morphophonology. Two proposals have 
been made to overcome this problem. 



Krieger et al. (93) encode finite state au- 
tomata directly in the feature formalism. Since 
two-level rules can be compiled into such au- 
tomata, morphophonology can be straightfor- 
wardly integrated into the grammar. While this 
is formally elegant it seems to be no good solu- 
tion for practical considerations. First, it is not 
entirely clear from their paper how the problem 
of null characters can be handled. Second, en- 
coding large automata will result in a very large 
and unwieldy type hierarchy. In general, introduc- 
ing automata into feature structures and encoding 
morphophonology directly at that level seems to 
be too low-level. 

Bird Hi Klein (93) argue against the use of 
two-level morphology because of linguistic con- 
siderations. The linguistic background of two- 
level rules — main stream segmental phonology — 
has widely been rejected as a valid linguistic 
model. Instead, they base their implementation 
on autosegmental phonology (cf. poldsmith (90|) ). 

This is certainly linguistically appealing. But 
there are reasons for sticking to a more conser- 
vative approach. Finite-state morphology as a 
formalism is not necessarily tied to segmental 
phonology. There are various approaches to cope 
with non-concatenative phenomena — one of them 
X2MorF ( |Trost, 90| ). Also, for a number of lan- 
guages complete sets of two-level rules do exist 
and can immediately be brought to bear. Finally, 
finite-state morphology has proven to be efficient 
while the method proposed by Bird fc Klein (9^ ) 
seems to be computationally costly. 

Like the other approaches ours is also based on 
HPSG. However, we employ a different approach 
to integration. Our grammar is encoded using a 
unification engine based on constraint logic pro- 
gramming (CLP). Besides conventional attribute- 
value descriptions this system allows for the direct 
representation of more general relations, as they 
are required by HPSG. This extension of the for- 
malism is used for the integration of morphology. 
Thus X2MorF is treated as one special relation 



of the grammar. As a result, our approach is more 
modular than the others. While being fully inte- 
grated morphology can still be viewed as an au- 
tonomous component leading to a more flexible 
design. 

We will now give an overview of X2MorF be- 
fore describing the integrated system and its im- 
plementation in detail. 



3 Word Level Processing — 

X2MORF 

X2MorF differs from standard two-level mor- 
phology in two important respects. Continua- 
tion classes are replaced by a feature-based word 
grammar. This allows for a more flne-grained 
description of morphs. It is also a prerequisite 
for a tight integration with a unification-based 
grammar. X2MorF uses a morph lexicon where 
each morph has one or more feature structures 
assigned. The word grammar itself is simple. 
Morphs have a functor-argument structure along 
the lines of di Sciullo & Williams (87). Affixes are 
unary functors while stems are arguments with- 
out any further structure, resulting in a binary 
tree structure. 

The other extension concerns the two-level 
rules, which are supplemented with a morpholog- 
ical filter consisting of a feature structure. This is 
important because in morphophonology only some 
rules are purely phonologically motivated. Oth- 
ers are triggered by a mixture of phonological and 
morphological facts. Such rules cannot be prop- 
erly represented in the standard approach. 

Take, e.g., umlaut and schwa epenthesis in 
German: The third person singular present tense 
suffix for German verbs is -t, e.g., sag-t — > sagt. 
For stems ending in a dental, schwa is inserted 
before the ending, e.g., bad-t — > badet. This rule 
does not hold across the whole vocabulary though. 
Stems of the strong paradigm do exhibit umlaut 
in 3rdPersSgPres which blocks schwa epenthesis. 
The final dental of the stem must be omitted in- 
stead, e.g., rat-t — > rat. 

The three ruled^ shown in Fig. ffl — together 
with the appropriate entries in the morph lexicon 
(cf. Fig. 1^ below) — produce the required behav- 
ior. In particular, these rules relate surface rat to 



^ These rules as well as other data presented in the ex- 
amples are simplified for the purpose of demonstration 



(i) A:d <=^ _ ; [morph|mhead|umlaut aou-umlaut] 
(ii) t:0 ^=^ _ +:0 t 
(iii) +:e <=^ dental _ +:0 [s \ t] ; [morph|mhead|epenthese +] 



Figure 1: Three extended two- level Rules 



lexical $rAt+t^. X2MorF can be seen as a re- 
lation between a surface string (the word form), a 
lexical string, and a feature structure (the inter- 
pretation of the word form) . Relevant for sentence 
level processing is the morphosyntactic informa- 
tion and the stem, found as the values of paths 
morph|mhead and MORPh|stem respectively (cf. 
Fig. ^ below). This is supplemented by lexeme 
specific information in the value of synsem (for a 



detailed description see [irost (ijijl )}' 



4 Implementing HPSG in a CLP 
Framework 

HPSG employs strongly typed feature structures 
together with principles constraining them fur- 
ther. Well-typedness requirements restrict the 
space of valid feature structures (cf. [Carpenter 
|(92| )): Every feature structure must be associated 
with a type, and every type restricts its associated 
feature structure in that only certain features are 
allowed and the values of these features must be 
of a certain type. Appropriateness and value re- 
strictions are inherited along the type hierarchy. 

The second source of constraints, in order to 
admit only linguistically valid feature structures, 
are the principles of grammar. Pollard fc Sag (87| ) 
allow general implicative and negative constraints 
in the form of conditional feature structures. In 
Pollard & Sag (94) principles are given only in 
verbal form. Recent work on formalizing the ba- 
sis of HPSG models them as constraints attached 



to types (e.g., Carpenter et al. (91 )). However, 
these distinctions affect only how the applicability 
of a principle is specified. More important for our 
present purpose is the form which the constraints 
expressed by a principle may take. Besides con- 
straints enforcing simple structure sharing (e.g., 
the Head Feature Principle given in Fig.|2|) there 
are also complex relational dependencies (e.g., in 



■^The lexical character A may have the surface realiza- 
tions a and a. The rule has an empty phonological context 
but a morphological filter. This is an example for the treat- 
ment of non-concatenative phenomena in X2MorF. 



the Subcategorization Principle^. Constraints 
like these go beyond the expressivity of pure fea- 
ture formalisms alone and need to be defined in a 
recursive manner. 

In order to integrate such complex constraints 
in the feature unification framework we interpret 
unification of typed feature structures under the 
restrictions of principled constraints as constraint 
solving in the CLP paradigm ( [laffar fc Lassez^ 

11)- 

In CLP the notion of unification is replaced 
by the more general notion of constraint solving. 
Constraint solvers may be embedded into a logic 
programming language either by writing a meta- 
interpreter or by making use of a system which 
allows for the implementation of unification ex- 
tensions. 

The second approach is taken by DMCAI 
CLP^ ( [Holzbaur, 92| ), a Prolog system whose uni- 
fication mechanism is extended in such a way 
that the user may introduce interpreted terms 
and specify their meaning with regard to unifica- 
tion through Prolog predicates. The basic mech- 
anism to achieve this behavior is the use of at- 
tributed variables, which may be qualified by ar- 
bitrary user-defined attributes. Attributed vari- 
ables behave like ordinary Prolog variables with 
two notable exceptions: when an attributed vari- 
able is to be unified with a non-variable term or 
another attributed variable the unification exten- 
sions come into play. For either case the user 
has to supply a predicate which explicitly specifies 
how the attributes interact and how they should 
be interpreted with respect to the semantics of 
the application domain. Unification succeeds only 
if these constraint solving clauses managing the 
combination — or verification — of the involved at- 
tributes are successful. 

The implementation of typed feature struc- 



■^"In a headed phrase (i.e., a phrasal sign whose dtrs 
value is of sort head-struc) , the SUBCAT value of the head 
daughter is the concatenation of the phrase's SUBCAT list 
with the list (in order of increasing obl iqueness) of SYNSE M 
values of the complement daughters." (Pollard & Sag, 94) 



"DMCAI CLP is an enhanced version of SICStus Prolog, 
available by anonymous ftp from ftp . ai . univie . ac . at 



tures in our system makes use of the CLP facilities 
provided by this enhanced Prolog system. Fea- 
ture structures are implemented by the attribute 
fs (Type, Dag, Goals), where Dag is a list of feature- 
value pairs (which is empty in case of atomic 
types) or a marker indicating uninstantiatedness 
of the substructure (feature structures are instan- 
tiated lazily). Goals is a list of delayed constraints 
(see below). Well- typed unification of two feature 
structures is implemented via the constraint solv- 
ing clauses mentioned above, taking into account 
type hierarchy and feature appropriateness (for a 
detailed description cf. Matiasek fc Heinz (931) ). 

Constraints imposed onto feature structures 
by the principles of grammar are stated in a con- 
ditional form where the antecedent is restricted to 
contain only typing requirements. p] In order to ac- 
count for these conditional constraints we adopt a 
licensing view: Every node of a feature structure 
has to be licensed by all principles of grammar. 

A node is licensed by a principle if either (i) 
the feature structure F rooted in that node sat- 
isfies the applicability conditions of the principle 
and the constraints expressed by the principle suc- 
cessfully unify with F, or (ii) the feature structure 
F rooted in that node is incompatible with the 
applicability conditions of the principle. The in- 
teresting case arises when a feature structure does 
not satisfy the applicability conditions of the prin- 
ciple but is compatible with them. Thus applica- 
bility of the principle can be decided only later, af- 
ter further instantiation or unification steps have 
restricted the (sub)structure rooted at that node. 
In precisely this case the application (or the aban- 
doning) of the constraint has to be delayed. The 
delay mechanism utilizes the Goals slot in the 
f s/3|^ attribute, which is dedicated to hold the 
delayed constraints. As an example take the well 
known Head Feature Principle of HPSG (Fig.^)|^ 
The conditional operator ===> is translated at 
read time via term_expansion/2 and implements 
the delay mechanism by compiling precondition 
checks into the principle. These antecedent checks 
trigger either the application of the principle, its 
abandonment, or its delay (by annotating the vari- 



^This is only a syntactic variant of attaching constraints 
solely to types (Carpenter et al., 91) a rid does not perm it 



11 



general conditional structures as used in Pollard & Sag I 
''pred/n is the usual notation for a n-ary Prolog 

predicate. 

^The operators : : = , : : , : , === are defined for typing 

of a node, path restriction, path concatenation and value 

restriction (type or coreference) respectively. 



AVM: 



headed- 
phrase 



synsem|loc|cat|head |T] 

dtrs|head-dtr|synsem|loc|cat|head|T] 



Prolog: 



head_f eature_priiiciple(X) :- 
X: :=headed_phrase 

X: : synsem: loc : cat :head===H, 

X : : dtrs : head_dtr : synsem : loc : cat : head===H . 



Figure 2: Head Feature Principle 

ables which are not sufficiently constrained to de- 
cide on the antecedent with the delayed goals). 

Two advantages of this approach to implement 
principled constraints are especially important for 
our present purpose: First, stating redundant 
typing requirements for embedded structures (i.e. 
type restrictions that would follow automatically 
from well-typing) forces delay of the conditional 
constraint until these substructures are instanti- 
ated. This device can, e.g., be used to block in- 
finite recursion in recursively defined constraints. 
Second, the right hand part of the conditional is 
not restricted to feature logical expressions, but 
instead can contain arbitrary Prolog goals. In 
this way constraints involving relational depen- 
dencies (such as the Subcategorization Principle 
and the morphological relation between a lexical 
and a surface string) can be expressed within the 
feature formalism and there is no need for external 
devices controlling this interaction. Furthermore, 
the conditional constraint syntax is not restricted 
to unary licensing principles but can also be used 
to express relations, such as f s_append/3 — needed 
for implementing the Subcat Principle — which ap- 
pends two feature structure lists (Fig. ^). Note 



f s_append(X, Y,Z) :- 

f s_empty_append(X,Y,Z) , 
f s_nonempty_appeiid (X , Y , Z) 

f s_empty_append(X,Y,Z) :- 
X: :=elist 
===> Y = Z. 

f s_nonempty_append(X,Y,Z) :- 



X 
===> X 

X 



=nelist 

first===F, Z : :f irst===F, 

rest===XRest , Z : :rest===ZRest , 



f s_append(XRest ,Y,ZRest) 



Figure 3: Append for feature structure lists 





t:0 ^=^ +:0 t 


Input 


_ <=> t:0 <=> ['+' :0, t:t] . 


Compiled 


morphrule([116,43,116|LS] , [Sc,48, 116 I SSO] ,SS,LCon,SCon,F) :- 
!, Sc=48, 

morphology([43,116|LS] , [48, 1161 SSO] ,SS, [116|LCon] , [HiSCon] ,F) . 



Figure 4: Sample Two-Level Rule 



morphology (LexStream, Surf StreamO, Surf Plainln.LexContext , Surf Context ,F) : 
instantiate (LexStream , Surf StreamO , Surf Plainin , Surf PlainOut , F) , 
morphrule (LexStream , Surf Stream , Surf PlainOut , LexContext , Surf Context , F) 



instantiate ([LC I LCs] , [SClSCs] , Surf Plainin, Surf PlainOut ,F) 
valid_alphabet_pair (LC,SC] , 

synchronize ([SClSCs] .SurfPlainln, Surf PlainOut) , 
lookahead (LC , LCs , SCs , Surf PlainOut ) . 



synchronize ( [48 I _] .Stream, Stream) :- !. 
synchronize ( [Char I _] , [Char I Stream] , Stream) . 



Figure 5: The morphology relation 



that disjunctive relations such as append can now 
be written as the conjunction of two specialized 
cases applying conditionally. Furthermore, in- 
finite loops due to uninstantiated variables can 
never occur, a crucial requirement when integrat- 
ing relational dependencies into a lazy instantiat- 
ing feature formalism. 



5 Embedding X2MorF into the 
Feature System 

Originally X2MorF was realized as a separate 
morphological component interfaced to the sen- 
tence analyzer/generator only via sequential data 
transfer. In the case of analysis, the feature struc- 
ture representing the word form was transmitted 
to the parser. For generation, X2MorF expected 
a feature structure as input reproducing one or 
more word forms. This purely sequential architec- 
ture was not satisfactory because of the problems 
mentioned in the introduction. 

In order to achieve tight integration, we adopt 
a relational view of X2MorF and encode the re- 
lation between surface string and lexical string di- 
rectly without using finite state automata (for ar- 



guments supporting this approach cf. Abramson 



(92)). However, our approach extends Abramson 
(92 ) in that it (i) explicitly accounts for the inser- 
tion of null characters and (ii) introduces the filter 
concept of X2MorF into the relational approach. 
The general format of a two-level rule specifi- 



cation in our system is 

LCon <=> Transition <=> RCon [ :- Filter/ 

in the case of equivalence rules, optional rules 
are written using only single arrows (=> and <=). 
These rules are compiled into Prolog clause^ re- 
lating the lexical and surface character streams 
appropriately (see Fig.^ for an example of the t- 
elision rule for German). 

To obtain a correct relationship between sur- 
face and lexical string every transition has to be 
licensed by a morphological rule. Transitions not 
mentioned by rules are handled by a default rule. 
Instantiation of contexts may not be done by the 
rules themselves, since this would make it impos- 
sible to obtain negation via the cut-operator. In- 
stead, it is handled separately in a backtrackable 
fashion. 

The central relation is the morphology predi- 
cate, (see Fig. ^) mediating between lexical string, 
surface string (with inserted null elements), the 
pure (denullified) surface string and the feature 
structure of the morphological sign. Instantiation 
of pairs is done depending on the possible lexi- 
cal continuations (the lexicon being represented 
by a trie-structure). The amount of lookahead is 
determined by the current pair which is to be li- 
censed by morphrule. R Synchronization of surface 

*Note that left contexts are encoded reversed to ac- 
count for the left to right traversal of the pair of character 
streams. Left contexts can be remembered and checked 
most efficiently this way. 

^This interaction and the lexicon lookup of the feature 



and lexical string by insertion of null characters is 
also handled at the instantiation level. 

The integration of the two-level relation into 
the general framework of the feature based 
sentence-level and word-level grammars is now 
performed by adding this relation as a principled 
constraint at the appropriate level. 

In a definite clause style AVM notation 
this could be written as follows (given that 
morphology/3 is a wrapper around the morphol- 
ogy relation given above, starting with empty left 
context and hiding the nullified surface stream): 



word 



PHON 



MORPII 



\2\3tring 

MSTRING [Tjsfnnj 
fs] STEM string 
MHEAD mhead 



msign 
HEAD head 
.SYNSEM synsem 



morp/io/off2/([T],[2],|l]) 



The actual implementation as a principled con- 
straint in our formalism additionally takes care of 
delaying the actual enforcement of this relation in 
case the strings are not sufficiently instantiated. 

A second provision has to be made in the 
word level grammar to assure proper concatena- 
tion of the lexical strings of the morphological 
signs being combined. Given the subtyping of 
msign into marg and mfunctor, which in turn 
has the subtypes leftfunctor and rightfunctor , the 
principled constraints ensuring concatenation of 
a left functor with its argument are shown in 
Fig. y. Concatenation is delayed until the ar- 



concat_right_f unctor (X) :- 
X: : =rightf unctor , 
X: : arg:mstring===subtype_of (string) 

X: : arg:mstring===Arg, 
X: :affix===Suffix, 
X: :mstring===Mstring, 
concat (Arg, Suffix, Mstring) . 



Figure 6: Concatenation of lexical strings 

gument's mstring is instantiated. Thus, infinite 
loops when concatenating are avoided. 

As an example we demonstrate how these con- 
straints interact in forming the third person sin- 
gular present tense form of the German verb raten 



structure corresponding to the current morph, which takes 
place when encountering a morph boundary is not shown 
for the sake of simphcity. 



(to guess). The lexical string is composed of the 
stem rAt and the suffix +t. The lexical entries of 
these two morphs are given in Fig. ^. 





"mstring "rAt" 






1 




STEM "rat" 






"epenthese - 






MHEAD 


PERSON 3 




marg 


[umlaut aou^umlaut 
verb^stem 


- 




STEM 1 string 




AFFIX "+t" 






EPENTHESE 3 boolean 

PERSON 3 






MHEAD 


TENSE tense_pres 








UMLAUT r2]umlaut 






verbjorm ' ' 






'stem 1 


rightfunctor 


ARG 

msign 


MHEAD 

- verb_stem 


EPENTHESE 
UMLAUT 


3 

2 



Figure 7: Lexical entries 

The two-level rules applicable for this example 
are the t- elision rule (Fig.^) and two rules with 
filters licensing a-umlaut and epenthesis, given in 
the input notation for our system (Fig.^). 

Interaction between syntactic and morpholog- 
ical processes takes place at the word level. The 
application of the two-level rules relating the sur- 
face string (i.e the PHON-value of the word) and 
the lexical-string (i.e. morph | mstring) is also 
triggered here. This interaction is completely neu- 
tral with respect to the direction of processing due 
to its relational nature. Parsing is performed by 
simply instantiating the PHON value. Generation 
can be achieved when morph | mstring is present, 
which in turn is obtained by concatenating the 
lexical strings of the msigns instantiated by the 
morph grammar. 

As a result of this constraint interaction the 
structure shown in Fig. ^ is obtained. Features 
relevant at the syntactic level (such as person 
and tense) are percolated from morph | mhead 
to synsem I LOG I cat I head via structure sharing 
constraints attached to the type word (this in- 
teraction is not shown in Fig. ^. Information 
on subcategorization and semantic content for the 
word is obtained from the lexeme lexicon using 
MORPH I STEM as a key. These constraints com- 
plete the interaction between syntactic and mor- 
phological processing at the word-level. 



A-umlaut 



<=> A:"a <=> _ :- filter(X, [X: :mhead:umlaut===aou_iimlaut] ) 



Epenthesis dental <=> ' + ':e <=> s_or_t :- filter(X, [X: :mhead:epenthese===' + '] ) 



Figure 8: Filter Rules 



N ' 


rat" 








"mstring 


"rAt+t" 




STEM 


T\"rat" 




AFFIX 


"-ht" 

'EPENTHESEfs]- 




MHEAD 


PERSON 3 

TENSE tense^pres 




verb- 


.UMLAUT 2 aou_umlaut_ 


PII 


form 


"MSTRING "rAt" 






STEM 


1 




ARC 


MHEAD 


epenthese| 3 1 

PERSON 3 






verb- 


UMLAUT 2 






L stem 





marg 



word 



right 
functor 

Figure 9: Result of constraint interaction 



6 Conclusion 

We have presented a framework for the tight inte- 
gration of word level and sentence level processing 
in a unification-based paradigm. The system is 
built upon a unification engine implemented in a 
CLP language supporting types and definite rela- 
tions as part of feature descriptions. Using this ex- 
tended feature formalism, which is independently 
motivated by requirements of standard HPSG, a 
reimplementation of X2MorF was integrated into 
the grammar as a specialized relation. 

This architecture has computational as well 
as linguistic advantages. Integrating morphology 
and morphophonology directly into the grammar 
is in the spirit of HPSG, which views grammar as a 
relation between the phonological (or graphemic) 
form of an utterance and its syntactic/semantic 
representation. This way the treatment of phe- 
nomena transcending the boundary between mor- 
phology and syntax is also made possible. 

On the implementation side, the practical 
problems of interfacing two inherently different 
modules are eliminated. For applications this 
means that using a morphological component is 
made easy. Nevertheless, this tight integration 
still leaves morphology and syntax/semantics as 
autonomous components, enabling direct use of 



existing data sets describing morphophonology in 
terms of the two-level paradigm. 
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